Systems and methods for determining defects in physical objects

ABSTRACT

In one embodiment, a method includes receiving, by a defect detector module, an image of a physical object and classifying, by the defect detector module, one or more first features from the image of the physical object into one or more first classifications using one or more machine learning algorithms. The method further includes analyzing, by the defect detector module, the one or more first classifications and determining, by the defect detector module, that the physical object comprises a defect based on analyzing the one or more first classifications.

PRIORITY

This application is a continuation under 35 U.S.C. § 120 of U.S. patentapplication Ser. No. 16/196,990 filed Nov. 20, 2018, which is a utilityfiling entitled “Systems and Methods for Determining Defects in PhysicalObjects”, which is hereby incorporated by reference in its entirety.

TECHNICAL FIELD

This disclosure generally relates to determining defects, and morespecifically to determining defects in physical objects.

BACKGROUND

Physical objects are used in various industries to perform theobjectives of business. For example, railcars move freight on tracks.The railcars and tracks include a combination of a number of physicalobjects. Over time, the physical objects may wear, break, or otherwisehave a defect, and the defect may require repair for continued, safeoperation of the railcars and tracks. Typically, physical objects suchas railway components are manually inspected by technicians to identifydefects.

SUMMARY

According to an embodiment, a method includes receiving, by a defectdetector module, an image of a physical object and classifying, by thedefect detector module, one or more first features from the image of thephysical object into one or more first classifications using one or moremachine learning algorithms. The method further includes analyzing, bythe defect detector module, the one or more first classifications anddetermining, by the defect detector module, that the physical objectcomprises a defect based on analyzing the one or more firstclassifications.

According to another embodiment, a system includes one or moreprocessors and a memory storing instructions that, when executed by theone or more processors, cause the one or more processors to performoperations including receiving, by a defect detector module, an image ofa physical object and classifying, by the defect detector module, one ormore first features from the image of the physical object into one ormore first classifications using one or more machine learningalgorithms. The operations further include analyzing, by the defectdetector module, the one or more first classifications and determining,by the defect detector module, that the physical object comprises adefect based on analyzing the one or more first classifications.

According to yet another embodiment, one or more computer-readablestorage media embody instructions that, when executed by a processor,cause the processor to perform operations including receiving, by adefect detector module, an image of a physical object and classifying,by the defect detector module, one or more first features from the imageof the physical object into one or more first classifications using oneor more machine learning algorithms. The operations further includeanalyzing, by the defect detector module, the one or more firstclassifications and determining, by the defect detector module, that thephysical object comprises a defect based on analyzing the one or morefirst classifications.

Technical advantages of certain embodiments of this disclosure mayinclude one or more of the following. The disclosed image capturingsystem automatically captures images of physical objects, whicheliminates or reduces manual data collection and human labor, savingtime and money. The image capturing system may capture images ofphysical objects such as rail components from angles that may not beidentified through manual inspection, which may increase accuracy indetecting defects in physical objects. The disclosed defect detectormodule may automatically detect defects in physical objects usingmachine learning algorithms, which may eliminate the manual labor ofscanning through and labeling images of potential defects. The systemsand methods described in this disclosure may be generalized to differenttransportation infrastructures, including rail, roads, and waterways.

Other technical advantages will be readily apparent to one skilled inthe art from the following figures, descriptions, and claims. Moreover,while specific advantages have been enumerated above, variousembodiments may include all, some, or none of the enumerated advantages.

BRIEF DESCRIPTION OF THE DRAWINGS

To assist in understanding the present disclosure, reference is now madeto the following description taken in conjunction with the accompanyingdrawings, in which:

FIG. 1 illustrates an example system for determining defects in physicalobjects;

FIG. 2 illustrates an example method for determining defects in physicalobjects;

FIG. 3 illustrates another example method for determining defects inphysical objects;

FIG. 4 illustrates an example image capturing system;

FIG. 5 illustrates an example module that may be used by the imagecapturing system of FIG. 4 ;

FIG. 6 illustrates an example method for tagging features with labels;and

FIG. 7 illustrates an example computer system that may be used by thesystems and methods described herein.

DETAILED DESCRIPTION

Physical objects such as railway components are used in variousindustries to perform the objectives of business. Over time, thephysical objects may break or have a defect, and the defect may requirerepair for continued, safe operation of the railcar. Typically, physicalobjects such as railway components are manually inspected by techniciansto identify defects.

Condition and defect analysis algorithms may utilize one or moretraditional computer vision techniques (e.g., region of interestanalysis, filtering, thresholding, blob techniques, edge analysis,contour extraction, histogram analysis, etc.) and common asset and/ordefect heuristics to automate detection of various conditions. Whilethese techniques may be effective in very specific and consistent cases,they are often brittle under real-world conditions and may be timeintensive to maintain, correct, and enhance.

The use of machine learning algorithms for image classification andsegmentation tasks includes a fundamentally different approach to imageanalysis than traditional computational methods. Rather thanalgorithmically filtering, distorting, grouping, segmenting, andcomputing on pixel matrices, machine learning algorithms (e.g., neuralnetworks) utilize a series of trained network layers through which animage or parts of an image pass through to make predictions about thecontent of the image.

Embodiments of this disclosure use deep machine learning to capture andanalyze visual imagery. The disclosed systems and methods recognizephysical objects (e.g., rail components) viewed from particular hardwareconfigurations on a vehicle in motion (e.g., a vehicle moving on a rail,a vehicle moving alongside a rail, etc.) and interprets visual databased on an amount, variety, and accuracy of labeled training data thatis continuously captured, pooled and weighted. The disclosed systems andmethods perform the interpretation by employing trained models on thecaptured image data for the physical objects to gain greater accuracyand higher-level analysis of the variety of physical objects captured.The machine learning algorithms are custom-trained for each hardwareconfiguration to achieve optimal continuous capture and analysis foreach variety of physical object. The algorithms are used to classifycertain features from images of the physical objects. Theclassifications are then used to determine defects in the physicalobjects.

FIGS. 1 through 7 show example systems and methods for determiningdefects in physical objects. FIG. 1 shows an example system fordetermining defects in physical objects. FIGS. 2 and 3 show examplemethods for determining defects in physical objects. FIG. 4 shows anexample image capturing system, and FIG. 5 shows an example module thatmay be used by the image capturing system of FIG. 4 . FIG. 6 shows anexample method for tagging features with labels. FIG. 7 shows an examplecomputer system that may be used by the systems and methods describedherein.

FIG. 1 illustrates an example system 100 for determining defects inphysical objects. System 100 of FIG. 1 includes a network 110, a defectdetector module 120, and an image capturing system 170. System 100 orportions thereof may be associated with an entity, which may include anyentity, such as a business, company (e.g., a railway company, atransportation company, etc.), or a government agency (e.g., adepartment of transportation, a department of public safety, etc.) thatmay determine defects in physical objects. The elements of system 100may be implemented using any suitable combination of hardware, firmware,and software.

Network 110 may be any type of network that facilitates communicationbetween components of system 100. Network 110 may connect defectdetector module 120 and image capturing system 170 of system 100.Although this disclosure shows network 110 as being a particular kind ofnetwork, this disclosure contemplates any suitable network. One or moreportions of network 110 may include an ad-hoc network, an intranet, anextranet, a virtual private network (VPN), a local area network (LAN), awireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), ametropolitan area network (MAN), a portion of the Internet, a portion ofthe Public Switched Telephone Network (PSTN), a cellular telephonenetwork, a 3G network, a 4G network, a 5G network, a Long Term Evolution(LTE) cellular network, a combination of two or more of these, or othersuitable types of networks. One or more portions of network 110 mayinclude one or more access (e.g., mobile access), core, and/or edgenetworks. Network 110 may be any communications network, such as aprivate network, a public network, a connection through Internet, amobile network, a WI-FI network, a Bluetooth network, etc. Network 110may include one or more network nodes. Network nodes are connectionpoints that can receive, create, store, and/or transmit data throughoutnetwork 110. Network 110 may include cloud computing capabilities. Oneor more components of system 100 may communicate over network 110. Forexample, defect detector module 120 may communicate over network 110,including receiving information from image capturing system 170.

Defect detector module 120 of system 100 represents any suitablecomputing component that may be used to determine defects 156 inphysical objects. Defect 156 is an imperfection that potentially impairsthe utility of the physical object. A physical object is an identifiablecollection of matter. Physical objects may include transportationinfrastructure components such as road, railway, airway, waterway,canal, pipeline, and terminal components. Examples of railway componentsinclude joints, switches, frogs, rail heads, anchors, fasteners, gageplates, ballasts, ties (e.g., concrete ties and wood ties), and thelike.

Defect detector module 120 includes an interface 122, a memory 124, anda processor 126. Interface 122 of defect detector module 120 representsany suitable computer element that can receive information from network110, transmit information through network 110, perform suitableprocessing of the information, communicate to other components (e.g.,components of image capturing system 170) of system 100 of FIG. 1 , orany combination of the preceding. Interface 122 represents any port orconnection, real or virtual, including any suitable combination ofhardware, firmware, and software, including protocol conversion and dataprocessing capabilities, to communicate through a LAN, a WAN, or othercommunication system that allows system 100 of FIG. 1 to exchangeinformation between components of system 100.

Memory 124 of defect detector module 120 stores, permanently and/ortemporarily, received and transmitted information, as well as systemsoftware, control software, other software for defect detector module120, and a variety of other information. Memory 124 may storeinformation for execution by processor 126. Memory 124 includes any oneor a combination of volatile or non-volatile local or remote devicessuitable for storing information. Memory 124 may include Random AccessMemory (RAM), Read-only Memory (ROM), magnetic storage devices, opticalstorage devices, or any other suitable information storage device or acombination of these devices. Memory 124 may include any suitableinformation for use in the operation of defect detector module 120.Additionally, memory 124 may be a component external to (or may bepartially external to) defect detector module 120. Memory 124 may belocated at any location suitable for memory 124 to communicate withdefect detector module 120. Memory 124 of defect detector module 120 maystore an image collection engine 130, a classification engine 132, adefect detector engine 134, a reporting engine 136, a labeling engine138, and a training engine 140. As another example, image collectionengine 130, classification engine 132, defect detector engine 134,reporting engine 136, labeling engine 138, and training engine 140 maybe external to memory 124 and/or defect detector module 120.

Image collection engine 130 of defect detector module 120 is anapplication that collects images 152 of physical objects. Imagecollection engine 130 may receive one or more images 152 of one or morephysical objects from image capturing system 170 via network 110. Imagecollection engine 130 may receive images 152 of physical objects inreal-time or near real-time as the physical objects are captured byimage capturing system 170. Image collection engine 130 may receiveimages 152 of physical objects in accordance with a schedule (e.g.,every minute, hour, week, etc.). Image collection engine 130 may combine(e.g., stitch together) one or more images 152 to create combined image152. Image collection engine 130 may group images 152 according to anysuitable combination such as by physical object, by a time image 152 wascaptured by image capturing system 170, by a time image 152 was receivedby image collection engine 130, and/or by a location 158 (e.g.,geographical location 158) where image 152 was captured.

Classification engine 132 of defect detector module 120 is anapplication that classifies features 154 from one or more images 152 ofone or more physical objects into one or more classifications 155. Eachfeature 154 may be a characteristic of image 152. For example, feature154 may include a railway component such as a bolt head. Feature 154 mayrepresent the physical object itself or one or more components of thephysical object. For example, first feature 154 may represent a physicalobject (e.g., a railway joint) and one or more second features 154 mayrepresent components of the railway joint (e.g., bolt heads, squarenuts, hex nuts, round nuts, holes, and the like.)

Classification engine 132 analyzes data from images 152 by applying oneor more machine learning algorithms. Machine learning algorithms may beassociated with one or more neural networks (e.g., a deep neuralnetwork), one or more deep learning algorithms, one or moreconvolutional neural networks (CNNs), artificial intelligence (AI), anyother suitable application, or any suitable combination of thepreceding. Each machine learning algorithm may be trained on labeledimage data to recognize one or more features 154 from one or more images152. One or more machine learning algorithms used by classificationengine 132 are trained to recognize specific physical objects (e.g.,rail components) as viewed by one or more components (e.g., one or morecameras) of image capturing system 170. For example, one or more CNNs166 may utilize a series of convolutional, pooling, reduction and fullyconnected layers through which image 152 or parts of image 152 passthrough to determine predictions about the content of image 152. One ormore machine learning algorithms output one or more classifications 155that describe one or more features 154 of one or more images 152 withina level of certainty. The level of certainty may depend on the number ofclassifications 155, the desired accuracy, the availability of sampletraining data for each classification 155, and the like. The level ofcertainty may be represented as a probability.

One or more classifications 155 are used to identify one or morefeatures 154. For example, first classification 155 may represent firstfeature 154 of a railway joint and one or more second classifications155 may represent one or more second features 154 associated with therailway joint such as bolt heads, square nuts, hex nuts, round nuts,holes, and the like. Classifications 155 may include locations offeatures 154 within image 152. For example, classification 155 for “bolthead” may include a location of the bolt head within image 152, whichmay be part of the physical object (e.g., a railway joint).

One or more classifications 155 may include one or more defects 156.Defects 156 include any attribute of a physical object that is animperfection or deficiency. Defects 155 may include imperfections suchas cracks, breaks, holes, mismatches, chips, wear, and the like. Forexample, defects 155 for a railway joint may include missing bolts, abreak, a crack, a tie condition (e.g., a deteriorated tie), a mismatchedrail, a rail end gap, rail end batter, a ballast condition (e.g.,deteriorated ballast), a right sized joint bar, and the like. As anotherexample, defects 156 for a railway switch may include a chipped point, abroken point, missing fasteners (e.g., stock rail), a skewed switch rod,a rail run at a switch point, inadequate tie spacing, inadequate rodspacing, inadequate flangeway width, and the like. Defects 156 for arailway frog may include a chipped point, a broken point, tread wear, abroken gage plate, a skewed gage plate, a missing fastener (e.g., amissing frog bolt), a missing plate, a broken guard rail, and the like.

Defects 156 for a railway head may include rail corrugation, railspalling, rail shelling, a broken rail, and the like. Defects 156 for arailway anchor and/or fastener may include a missing anchor and/orfastener, an inadequate anchor pattern, missing clips, missing spikes,and the like. Defects 156 for a railway gage plate may include a skewedgage plate, a broken gage plate, and the like. Defects 156 for railwayballast may include insufficient ballast, dirty ballast, fouled ballast,and the like. Defects 156 for a railway concrete tie may include abroken tie, a cracked tie, a deteriorated tie, an insufficient distancebetween ties, and the like. Defects 156 for a railway wood tie mayinclude a plate cut, a wheel cut, an insufficient distance between ties,a rotted tie, a hollow tie, and the like. Other defects 156 may beidentified or defined based on the physical object being captured.

Defect detector engine 134 of defect detector module 120 is anapplication that determines whether a physical object includes one ormore defects 156 based on analyzing output of the one or more machinelearning algorithms (e.g., CNNs 166). For example, defect detectorengine 134 may analyze one or more classifications 155 output by one ormore machine learning algorithms. Defect detector engine 134 may applyone or more algorithms to the detection results (e.g., classifications155) to determine (e.g., optically identify) one or more defects 156.

Defect detector engine 134 may analyze one or more classifications 155to identify one or more defects 156. For example, defect detector engine134 may determine that a physical object (e.g., a railway joint)includes defect 156 if classification 155 associated with the physicalobject is defect 156 (e.g., a break). Defect detector engine 134 maydetermine location 158 of defect 156 on the physical object relative toother features 154 of the physical object to produce additionalinformation about defect 156. For example, defect detector engine 134may identify a break in a joint bar based on the results of one or moremachine learning algorithms including classifications 155 for “break”and “joint bar.” Defect detector engine 134 may determine that the jointbar includes bolts based on classification 155 for “bolts.” Defectdetector engine 134 may determine location 158 of the break relative tothe bolts. Defect detector engine 134 may then produce additionalinformation about the break based on its relative location 158. Forexample, defect detector engine 134 may classify the break as “centerbroken defect” if the break is between two middle bolts. As anotherexample, defect detector engine 134 may classify the break as “quarterbroken defect” if the break is outside the two middle bolts.

Defect detector engine 134 may determine geographical location 158 ofdefect 156 of the physical object. Defect detector engine 134 maydetermine geographical location 158 using information received fromimage collection engine 130. Image collection engine 130 may captureinformation (e.g., a latitude and a longitude) representing one or moregeographical locations 158 associated with one or more images 152 andtransmit this information to defect detector engine 134. Defect detectorengine 134 may translate the information received from image collectionengine 130 into any suitable format. For example, defect detector engine134 may translate a latitude and longitude received from imagecollection engine 130 into a track type, a track number, a line segment,milepost information, and the like. Defect detector engine 134 may useone or more algorithms (e.g., a closest point algorithm) to locategeographical location 158 on a map (e.g., an existing map of a railroadtrack).

In certain embodiments, defect detector engine 134 may detect welds(e.g., thermite welds) in images 152 of rails and use the weld locationsto identify mileposts. Each image 152 may be “n” feet (e.g., five feet)in width, representing “n” feet (e.g., five feet) of rail, where nrepresents any suitable number. Defect detector engine 134 may determinegeographical location 158 of the physical object by determining thefootage from image 152 of the physical object relative to a milepost.Defect detector engine 134 may determine geographical location 158 ofthe physical object by footage from the physical object in an ascendingand/or descending milepost direction. For example, defect detectorengine 134 may determine that the physical object is a certain number offeet (e.g., 1000 feet) past mile post 100, which may be represented(e.g., output) as “MP 100.0+1000 Feet.” As another example, defectdetector engine 134 may determine that the physical object is a certainnumber of feet (e.g., 1000 feet) prior to mile post 137, which may berepresented (e.g., output) as “MP 100.0-1000 Feet.” In certainembodiments, defect detector engine 134 may represent geographicallocation 158 as Global Positioning System (GPS) coordinates.

Defect detector engine 134 may determine geographical location 158 basedat least in part on location 158 of defect 156 relative to the physicalobject. For example, defect detector engine 134 may determine ageographical location 158 of a break of a rail joint based in part onlocation 158 of the break relative to the rail joint. Defect detectorengine 134 may determine geographical location 158 based at least inpart on one or more sensors (e.g., a position sensor) located withinsystem 100 (e.g., image capturing system 170.) Geographical location 158may be defined by coordinates (e.g., longitude and latitude.)

Defect detector engine 134 may also determine an identification mark ofthe physical object. The identification mark of the physical object mayinclude any characters (e.g., numbers, letters, etc.) suitable toidentify the physical object. The identification mark of the physicalobject may identify an owner and/or a manufacturer of the physicalobject (e.g., a railroad manufacturer). The identification mark of thephysical object may be used to trace the physical object to its origin.For example, the identification mark may include a batch code thatallows the physical object to be traced back to a specific manufacturingbatch. Defect detector engine 134 may determine an identification markof the physical object based on geographical location 158 of thephysical object and/or by analyzing one or more images 152 of thephysical object.

Reporting engine 136 of defect detector module 120 is an applicationthat generates one or more reports 160. Reporting engine 136 maygenerate report 160 indicating that the physical object is not defectiveif defect detector engine 134 determines that the physical object doesnot include defect 156. Reporting engine 136 may generate report 160indicating that the physical object is defective if defect detectorengine 134 determines that the physical object includes defect 156.Reports 160 may be in any suitable form (e.g., written and/or verbal) tocommunicate defects 156. Reports 160 may include information associatedwith the physical objects, features 154, classifications 155, locations158, labels 162, and the like. For example, reports 160 may includelabels 162 shown in FIG. 6 , locations 158 (e.g., geographical locations158), and/or identification marks. Reports 160 may include one or morediagrams, tables, lists, graphs, and/or any other suitable format forcommunicating information. Reporting engine 136 may communicate reports160 to one or more components of system 100 (e.g., image capturingsystem 170) and/or a user (e.g., an administrator, a technician, etc.)of system 100.

Labeling engine 138 of defect detector module 120 is an application thatgenerates one or more labels 162. Labels 162 provide visual informationassociated with features 154, classifications 155, defects 156, and/orlocations 158. Labeling engine 138 tags features 154 of one or morephysical objects with labels 162. Each label 162 may include informationassociated with the physical object. Each label 162 may include features154, probabilities, sizes (e.g., diameters, areas, and the like)associated with features 154, classifications 155, locations 158, andthe like. For example, label 162 may represent feature 154 as “Bolt-head(0.77)” to describe classification 155 of “bolt-head” and a probabilityof 0.77 (i.e., 77 percent) that feature 154 is accurately identified asa bolt head.

Labeling engine 138 may insert one or more portions of label 162representative of feature 154 of a physical object (e.g., a rail joint)on a diagram of the physical object. Labeling engine 138 may insert oneor more portions of label 162 on the diagram at location 158 thatassociated feature 154 occurs on the physical object. For example, forfeature 154 of a bolt head, labeling engine 138 may insert one or moreportions of label 162 on a diagram of the physical object at location158 where the bolt head is positioned on the physical object. Labels 162may represent features 154 with any suitable shape or character. Forexample, label 162 representative of feature 154 of a bolt head may be abounding box having a square or rectangular shape. Defect detectorengine 134 may use labels 162 to further classify defects 156, asdescribed in FIG. 6 below.

Training engine 140 of defect detector module 120 is an application thattrains one or more machine learning algorithms (e.g., CNNs 166).Training engine 140 trains machine learning algorithms using trainingdata (e.g., training images 164) by which weights may be adjusted toaccurately recognize a physical object. Machine learning algorithms maybe trained on each of the specific camera angles of image capturingsystem 170 to provide the highest accuracy for each classification 155.Training engine 140 may train one or more machine learning algorithms bycollecting sample data (e.g., training images 164) representative ofeach classification 155. Training engine 140 may label the sample datawith labels 162 and use the labeled data to train one or more machinelearning algorithms to recognize each classification 155. Trainingengine 140 may use a subset of the labeled imagery to check the accuracyof each machine learning algorithm.

Training engine 140 may receive initial training data from anadministrator of system 100. The amount and variety of the training data(e.g., training images 164) utilized by one or more machine learningalgorithms depends on the number of classifications 155, the desiredaccuracy, and the availability of sample data for each classification155.

Memory 124 may store database 150. Database 150 may store certain typesof information for defect detector module 120. For example, database 150may store images 152, features 154, classifications 155, defects 156,locations 158, reports 160, labels 162, training images 164, and machinelearning algorithms (e.g., CNNs 166). Database 150 may be any one or acombination of volatile or non-volatile local or remote devices suitablefor storing information. Database 150 may include RAM, ROM, magneticstorage devices, optical storage devices, or any other suitableinformation storage device or a combination of these devices. Database150 may be a component external to defect detector module 120. Database150 may be located in any location suitable for database 150 to storeinformation for defect detector module 120. For example, database 150may be located in a cloud environment.

Processor 126 of defect detector module 120 controls certain operationsof defect detector module 120 by processing information received frominterface 122 and memory 124 or otherwise accessed by processor 126.Processor 126 communicatively couples to interface 122 and memory 124.Processor 126 may include any hardware and/or software that operates tocontrol and process information. Processor 126 may be a programmablelogic device, a microcontroller, a microprocessor, any suitableprocessing device, or any suitable combination of the preceding.Additionally, processor 126 may be a component external to defectdetector module 120. Processor 126 may be located in any locationsuitable for processor 126 to communicate with defect detector module120. Processor 126 of defect detector module 120 controls the operationsof image collection engine 130, classification engine 132, defectdetector engine 134, reporting engine 136, labeling engine 138, andtraining engine 140.

One or more components of defect detector module 120 may operate in acloud. The cloud may deliver different services (e.g., applications,servers, and storage) to defect detector module 120 through network 110.The cloud may be implemented using any suitable combination of hardware,firmware, and software. For example, the cloud may be implemented usingone or more components of the computer system of FIG. 7 .

Image capturing system 170 of system 100 represents a system forcapturing images 152 of physical objects. Image capturing systemincludes a sub-frame 172, a beam 174, one or more modules 176, alighting system 178, a heating, ventilation, and air conditioning (HVAC)system 180, a data system 182, and one or more controllers 184. Thecomponents of image capturing system 180 may be attached (e.g.,physically attached) to a vehicle (e.g., a locomotive). Image capturingsystem 170 may capture one or more images 152 of a physical object whilethe vehicle is in motion relative to the physical object. For example,image capturing system 170 may be attached to a locomotive, and imagecapturing system 170 may capture images 152 of rail joints while thelocomotive travels along a railroad track. Image capturing system 170may operate similar to a flatbed document scanner with the exceptionthat image capturing system 170 is in motion while capturing images 152of stationary physical objects. Image capturing system 170 is describedin more detail in FIG. 4 below.

Although FIG. 1 illustrates a particular arrangement of network 110,defect detector module 120, interface 122, memory 124, processor 126,image collection engine 130, classification engine 132, defect detectorengine 134, reporting engine 136, labeling engine 138, training engine140, database 150, and image capturing system 170, this disclosurecontemplates any suitable arrangement of network 110, defect detectormodule 120, interface 122, memory 124, processor 126, image collectionengine 130, classification engine 132, defect detector engine 134,reporting engine 136, labeling engine 138, training engine 140, database150, and image capturing system 170. Network 110, defect detector module120, interface 122, memory 124, processor 126, image collection engine130, classification engine 132, defect detector engine 134, reportingengine 136, labeling engine 138, training engine 140, database 150, andimage capturing system 170 may be physically or logically co-locatedwith each other in whole or in part.

Although FIG. 1 illustrates a particular number of networks 110, defectdetector modules 120, interfaces 122, memories 124, processors 126,image collection engines 130, classification engines 132, defectdetector engines 134, reporting engines 136, labeling engines 138,training engines 140, databases 150, and image capturing systems 170,this disclosure contemplates any suitable number of networks 110, defectdetector modules 120, interfaces 122, memories 124, processors 126,image collection engines 130, classification engines 132, defectdetector engines 134, reporting engines 136, labeling engines 138,training engines 140, databases 150, and image capturing systems 170.One or more components of defect detector module 120 and/or imagecapturing system 170 may be implemented using one or more components ofthe computer system of FIG. 7 .

Although FIG. 1 describes system 100 for determining defects 156 inphysical objects, one or more components of system 100 may be applied toother implementations. For example, one or more components of defectdetector module 120 and/or image capturing system 170 may be utilizedfor asset identification and/or inventory. For example, classificationengine 132 of defect detector module 120 may be used to identifyphysical objects (e.g., rail joint bars, switches, crossings, frogs,etc.) and record inventory of identified physical objects.

In operation, image collection engine 130 of defect detector module 120receives image 152 of a physical object (e.g., a rail joint) from imagecapturing system 170. Classification engine 132 of defect detectormodule 120 classifies features 154 (e.g., bolt, break, discontinuity,and hole) from image 152 of the physical object into classifications 155(e.g., bolt, break, discontinuity, and hole) using one or more machinelearning algorithms (e.g., CNNs 166). Defect detector engine 134 ofdefect detector module 120 analyzes classifications 155 and determinesthat the physical object includes defect 156 (e.g., break) based on theanalysis. Defect detector engine 134 determines location 158 of defect156 relative to other features 154 of the physical object using image152. Defect detector engine 134 determines geographical location 158 ofdefect 156 of the physical object based at least in part on location 158of defect 156 relative to other features 154 of the physical object.Reporting engine 136 generates report 160 indicating that the physicalobject includes defect 156. Report 160 includes geographical location158 of defect 156. Labeling engine 138 labels one or more first features154 of image 152 with one or more labels 162. Labels 162 include label162 representing defect 156 (e.g., break) of the physical object.

As such, system 100 of FIG. 1 determines defect 156 in a physical objectby capturing images 152 of the physical object, analyzing images 152,classifying features 154 from images 152 using one or more machinelearning algorithms, and determining defect 156 based on classifications155, which reduces or eliminates the need for manual inspection.

FIG. 2 shows an example method 200 for determining defects in physicalobjects. Method 200 begins at step 210. At step 220, a defect detectormodule (e.g., defect detector module 120 of FIG. 1 ) receives an image(e.g., image 152 of FIG. 1 ) of a physical object from an imagecapturing system (e.g., image capturing system 170 of FIG. 1 ). Forexample, an image collection engine (e.g., image collection engine 130of FIG. 1 ) of the defect detector module may receive an image of a railjoint from the image capturing system. The image capturing system may beattached to a component (e.g., a locomotive), and the image capturingsystem may capture the image while the component is in motion relativeto the physical object.

At step 230, the defect detector module classifies one or more features(e.g., features 154 of FIG. 1 ) from the image into one or moreclassifications (e.g., classifications 155 of FIG. 1 ) using one or moremachine learning algorithms (e.g., CNNs 166 of FIG. 1 ). For example, aclassification engine (e.g., classification engine 132) of the defectdetector module may use one or more CNNs to classify features such as asquare nut, a bolt head, a break, a hole, a discontinuity, and a barinto classifications that include a square nut, a bolt head, a break, ahole, a discontinuity, and a bar, respectively. One or more algorithmsmay logically reduce the square nut and bolt head classifications to asingle bolt classification since the visual distinction between a squarenut and a bolt head equates to the physical representation of a bolt.

At step 240, the defect detector module analyzes the one or moreclassifications. A defect detector engine (e.g., defect detector engine134 of FIG. 1 ) of the defect detector module may analyze the one ormore classifications to identify the presence of a defect (e.g., defect156 of FIG. 1 ) in the physical object. At step 250, the defect detectormodule determines whether the physical object is defective based on theclassifications. The defect detector engine may determine whether thephysical object is defective if the one or more classifications includea defect (e.g., a break).

If the defect detector module determines that the physical object is notdefective based on the classifications, method 200 advances from step250 to step 260, where a reporting engine (e.g., reporting engine 136 ofFIG. 1 ) generates a report indicating that the physical object is notdefective. Method 200 then moves to step 280, where method 200 ends. Ifthe defect detector module determines that the physical object isdefective based on the classifications, method 200 advances from step250 to step 270, where the reporting engine generates a reportindicating that the physical object is defective. Method 200 then movesto step 280, where method 200 ends.

Modifications, additions, or omissions may be made to method 200depicted in FIG. 2 . Method 200 may include more, fewer, or other steps.For example, method 200 may include training, by a training engine(e.g., training engine 140 of FIG. 1 ) of a defect detector module, oneor more neural networks and/or algorithms (e.g., CNNs) to correctlyrecognize and classify features within images. As another example,method 200 may include generating, by a labeling engine (e.g., labelingengine 138 of FIG. 1 ) of defect detector module, one or more labels(e.g., labels 162 of FIG. 1 ) representative of the one or morefeatures. Steps may be performed in parallel or in any suitable order.While discussed as specific components completing the steps of method200, any suitable component may perform any step of method 200.

FIG. 3 shows an example method 300 for determining defects in physicalobjects. Method 300 begins at step 305. At step 310, an image collectionengine (e.g., image collection engine 130 of FIG. 1 ) of a defectdetector module (e.g., defect detector module 120 of FIG. 1 ) receivesan image (e.g., image 152 of FIG. 1 ) of a physical object from an imagecapturing system (e.g., image capturing system 170 of FIG. 1 ). Theimage collection engine of the defect detector module may detect thepresence of a rail joint in the image.

At step 315, the defect detector module classifies a first feature(e.g., feature 154 of FIG. 1 ) from the image into a firstclassification (e.g., classification 155 of FIG. 1 ). For example, oneor more CNNs may be trained to detect classifications such as “bar,”“discontinuity,” and “end post,” and a classification engine (e.g.,classification engine 132 of FIG. 1 ) of the defect detector module mayclassify a first feature of a bar as a “bar” classification using theone or more CNNs. At step 320, the defect detector module crops theimage to an area surrounding the first feature. For example, theclassification engine may crop the image to an area surrounding the barof the rail joint.

At step 325, the defect detector module classifies one or more secondfeatures from the image into one or more second classifications usingthe one or more CNNs. For example, one or more CNNs may be trained todetect second classifications such as “bolt head,” “square nut,” “hexnut,” “round nut,” “hole,” and “break,” and the classification engine ofthe defect detector module may classify a second feature of a break as a“break” classification using the one or more CNNs.

At step 330, the defect detector module analyzes the one or more secondclassifications. At step 335, the defect detector module determineswhether the image includes a defect (e.g., defect 156 of FIG. 1 ) basedon analyzing the one or more second classifications. For example, adefect detector engine (e.g., defect detector engine 134 of FIG. 1 ) ofthe defect detector module may determine that the image of the physicalobject includes a defect (e.g., defect 156 of FIG. 1 ) if the one ormore second classifications is a defect.

If the defect detector module determines that the image does not includea defect based on the one or more second classifications, method 300advances from step 335 to step 345, where a reporting engine (e.g.,reporting engine 136 of FIG. 1 ) generates a report indicating that thephysical object is not defective. Method 300 then moves from step 345 tostep 365, where method 300 ends.

If the defect detector module determines that the image includes adefect based on the one or more second classifications, method 300advances from step 335 to step 340. At step 340, the defect detectormodule determines whether the defect is part of the physical object. Forexample, the defect detector engine may then determine that the physicalobject includes a defect if the location of the defect is part of thephysical object of the image. As another example, the defect detectorengine may determine that the physical object does not include a defectif the location of the defect is outside of the physical object of theimage (e.g., part of a different physical object of the same image). Ifthe defect detector module determines that the defect is not part of thephysical object, method 300 advances from step 340 to step 345, wherethe reporting engine generates a report indicating that the physicalobject is not defective. Method 300 then moves from step 345 to step365, where method 300 ends.

If the defect detector module determines that the defect is part of thephysical object, method 300 advances from step 340 to step 350, wherethe defect detector module determines a location of the defect in thecropped image. At step 355, the defect detector module determines ageographical location (e.g., a GPS location) of the defect of thephysical object using the location of the defect within the image andsensor information obtained from an image capturing system (e.g., imagecapturing system 170 of FIG. 1 ). Method 300 then advances to step 360where the reporting engine generates a report indicating that thephysical object is defective. The report may also indicate thegeographical location of the defect of the physical object. Method 300then moves from step 360 to step 365, where method 300 ends.

Modifications, additions, or omissions may be made to method 300depicted in FIG. 3 . Method 300 may include more, fewer, or other steps.For example, method 300 may include further classifying the defect(e.g., a break) of the physical object (e.g., a rail joint bar) into amore specific defect (e.g., a center broken joint bar or a quarterbroken joint bar) based on a location of the defect relative to otherfeatures (e.g., bolts, holes, and discontinuities) of the physicalobject. Steps may be performed in parallel or in any suitable order.While discussed as specific components completing the steps of method300, any suitable component may perform any step of method 300.

FIG. 4 shows an example image capturing system 170. Image capturingsystem 170 captures images (e.g., images 152 of FIG. 1 ) of physicalobjects and communicates the images to a defect detector module (e.g.,defect detector module 120 of FIG. 1 ). Image capturing system 170 orportions thereof may be associated with an entity, which may include anyentity, such as a business, company (e.g., a railway company, atransportation company, etc.), or a government agency (e.g., adepartment of transportation, a department of public safety, etc.) thatcaptures images. The elements of system 100 may be implemented using anysuitable combination of hardware, firmware, and software.

Image capturing system 170 of FIG. 4 includes a vehicle 410, a rotaryposition encoder 420, a sub-frame 172, and a beam 174. Vehicle 410 isany machine to which beam 174 may be connected. Vehicle 410 may have anengine and/or wheels. Vehicle 410 may be a car, a locomotive, a truck, abus, an aircraft, or any other machine suitable for mobility. Vehicle410 may operate at any speed that allows one or more components (e.g.,sensors, cameras, etc.) of beam 174 to capture images. For example,vehicle 410 may be a rail bound vehicle that travels 70 miles per hour.

Rotary position encoder 420 of image capturing system 170 is a wheelencoder or other timing device used to measure axle rotation. Rotaryposition encoder 420 may measure the number of times an axle makes arevolution. Rotary position encoder 420 may be attached to an axle ofvehicle 410. Rotary position encoder 420 may be physically and/orlogically connected to one or more components of image capturing system170. For example, rotary position encoder 420 may be physically and/orlogically connected to one or more cameras and/or sensors of modules176. As another example, rotary position encoder 420 may be physicallyand/or logically connected to controller 184.

Rotary position encoder 420 may communicate with a camera of module 176to ensure that the camera captures images of the same perspective andproportion regardless of the speed of travel of vehicle 410. Forexample, rotary position encoder 420 may be synchronized with multiplecameras of beam 174 to ensure that all cameras are taking images at thesame time. As another example, rotary position encoder 420 may besynchronized with a camera of beam 174 to ensure that a camera travelingwith vehicle 410 at a first speed (e.g., 7 miles per hour) capturesimages that are the same perspective and proportion of a cameratraveling with vehicle 410 at a second speed (e.g., 70 miles per hour).

Sub-frame 172 of image capturing system 170 is an intermediate structureconnecting vehicle 410 to beam 174. Sub-frame 172 engages vehicle 410 ata plurality of locations. Sub-frame 172 may be connected to vehicle 410and/or beam 174 with one or more bolts 430, welds, and/or any othersuitable coupling. Slots 440 of sub-frame 172 provide level and/orheight adjustments for beam 174. Slots 440 may be vertical and/orhorizontal. Vertically oriented slots 440 of sub-frame 172 provideheight adjustments for beam 174. Sub-frame 174 may be connected to afront end of vehicle 410, a back end of vehicle 410, a side of vehicle410, or any other suitable location to connect vehicle 410 to beam 174.Sub-frame 172 may be made of metal (e.g., steel or aluminum), plastic,or any other material suitable to connect vehicle 410 and beam 174. Incertain embodiments, sub-frame 172 may be omitted such that beam 174attaches directly to vehicle 410.

Beam 174 of image capturing system 170 is a structure that contains andorients components (e.g., cameras and sensors) used to capture images.In certain embodiments, beam 174 operates similar to a flatbed documentscanner with the exception that beam 174 is in motion while capturingimages 152 of stationary physical objects. Beam 174 engages withsub-frame 172. For example, beam 174 may be bolted to sub-frame 172 withbolts 430. In the illustrated embodiment of FIG. 4 , beam 174 has threesections that include two end sections and a center section. Beam 174has a gullwing configuration such that the center section bends inwardtoward the center of beam 174. The gullwing configuration allows theimage capturing components (e.g., sensors, cameras, etc.) of modules 176within beam 174 to be properly oriented within with respect to thephysical objects being captured. In certain embodiments, the centersection of beam 174 may be omitted, and each end section is connected tosub-frame 172. Beam 174 may be made of metal (e.g., steel or aluminum),plastic, or any other material suitable for housing components of beam174 and for attaching beam 174 to sub-frame 172.

Beam 174 may include one or more openings 450. Openings 450 may providefor the placement of modules 176 within beam 174. Openings may allow forinstallation, adjustment, and maintenance of modules 176. While beam 174is illustrated in FIG. 4 as having a particular size and shape, beam 174may have any size and shape suitable to house and orient its components(e.g., modules 176). Other factors that may contribute to the design ofbeam 174 include shock resistance, vibration resistance, weatherproofingconsiderations, durability, ease of maintenance, calibrationconsiderations, and ease of installation.

Beam 174 includes modules 176, lighting system 178, HVAC system 180,data system 182, and controller 184. Modules 176 house components forcapturing images. Each module 176 may include one or more sensors,cameras, and the like. Modules 176 are located within beam 174 andassist with positioning and supporting the image capturing componentswithin beam 174. Modules 176 are designed to allow for serviceabilityand adjustment.

In certain embodiments, each end section of beam 174 houses one or morecamera modules 176. For example, a first end section of beam 174 mayhouse module 176 that includes two downward facing cameras that captureimages of tie and ballast areas of a rail. The first end section of beam174 may house the two downward facing cameras in a portion of the firstend section that is substantially horizontal to the rail. The second endsection of beam 174 opposite the first end section may house two modules176 that each include two angled cameras that capture images of bothsides of the rail and rail fastening system. The second end section ofbeam 174 may house the four angled cameras in portions of the second endsection that are at an angle (e.g., a 45 degree angle) to the rail.

Modules 176 of image capturing system 170 may house various types ofsensors depending on sensing and/or measuring requirements. Sensorshoused by modules 176 may include optical sensors (e.g., cameras forvisible light (mono and color), infrared, UltraViolet, and/or thermal),motion sensors (e.g., gyroscopes and accelerometers), light detectionand ranging (LIDAR) sensors, hyperspectral sensors, GPS sensors, and thelike. Optical sensors and lasers may be used together for lasertriangulation to measure deflection or profile. LIDAR sensors may beused for generating three-dimensional (3D) point-cloud data.Hyperspectral sensors may be used for specific wavelength responses. Anexample module 176 is described in FIG. 5 below.

Lighting system 178 of image capturing system 170 provides externalillumination for capturing images. Lighting system 178 providesillumination for capturing images during daylight and darkness. Lightingsystem 178 may provide lighting intensity sufficient to capture imagesof stationary physical objects while lighting system 178 travels at apredetermined speed (e.g., 70 miles per hour.) Lighting system 178 mayinclude mechanisms for properly orienting the illumination. Lightingsystem 178 may include any type of lights suitable to provide adequateillumination for capturing images. Lighting system 178 may include lightemitting diode (LED) lights (e.g., white LED lights, off-road racing LEDlights, etc.), light bars (e.g., off-road racing LED light bars),auxiliary lights (e.g., LED auxiliary lights), infrared lighting, acombination of these, or any other suitable types of lights. Lightingsystem 178 may include one or more components that provide laserillumination, infrared illumination, ultraviolet illumination, or anyother suitable type of illumination required by one or more components(e.g., sensors, cameras, etc.) of modules 176.

HVAC system 180 of image capturing system 170 provides heating,ventilation, and/or air conditioning to beam 174 of image capturingsystem 170. HVAC system 180 may regulate environmental conditions (e.g.,an internal temperature, humidity, etc.) of beam 174. HVAC system 180may monitor the environmental conditions of beam 174 to ensure thatoperating requirements of modules 176 of beam 174 are satisfied. Forexample, HVAC system 180 may provide cooling to beam 174 to ensure atempered environment for modules 176 (e.g., cameras and sensors) tooperate during hot weather. HVAC system 180 may be a traditional HVACsystem that includes one or more of a condenser, an evaporator, acompressor, an expansion valve, a belt, a hose, refrigerant, and thelike.

HVAC system 180 may provide cooling to beam 174 via an air-poweredvortex generator. Dried, filtered, and regulated air from a mainreservoir system of vehicle 410 (e.g., a locomotive) may be applied tothe vortex generator. The vortex generator may apply cooled, compressedair into beam 174. The cooled air may be routed to each module 176(e.g., one or more sensors) for direct application. The vortex generatormay also operate as a humidity regulator. For example, the vortexgenerator may regulate humidity to approximately 50 percent. thecompressed air may provide a slight positive pressure on beam 174, whichmay prevent external dust and/or debris from entering beam 174 throughsmall holes.

Data system 182 of image capturing system 170 connects and directs alldata (e.g., sensor data) received from one or more components of imagecapturing system 170 to one or more computers, one or more controllers(e.g., controller 184), and/or one or more storage devices. Data system182 may include one or more data cables for communicating the data tothe controller and/or storage device. The data cables of data system 182may include internal cables that reside inside beam 174. The data cablesof data system 182 may include external cables that reside outside beam174. The internal and external cables may be joined by a weatherproofconnector located at a wall of beam 174. One or more external cables maybe routed to the one or more computers, the one or more controllers,and/or the one or more storage devices. Data cables of data system 182provide a path for data ingress and/or egress. For example, data cablesof data system 182 may trigger signal ingress from controller 184. Asanother example, data system 182 may communicate data to controller 184and/or a storage device wirelessly using any suitable wireless orcellular communication protocol.

Controller 184 of image capturing system 170 represents any suitablecomputing component that may be used to process information for imagecapturing system 170. Controller 184 may coordinate one or morecomponents of image capturing system 170. Controller 184 may receivedata from modules 176, lighting system 179, HVAC system 180, data system182, vehicle 410, and/or rotary position encoder 420. Controller 184 maymonitor inputs and/or outputs of modules 176, lighting system 178, HVACsystem 180, and rotary position encoder 420. Controller 184 may includea communications function that allows users (e.g., a technician) toengage image capturing system 170 directly. For example, controller 184may be part of a computer (e.g., a laptop), and a user may accesscontroller 184 through an interface (e.g., a screen, a graphical userinterface (GUI), or a panel) of the computer. Controller 184 maycommunicate with one or more components of image capturing system 170via a network (e.g., network 110 of FIG. 1 ). For example, controller184 may communicate with one or more components of image capturingsystem 170 via an LAN and/or from a remote terminal on a WAN such as acellular or wireless network. Controller 184 may be located insidevehicle 410.

Controller 184 may initiate adjustments to one or more components ofimage capturing system 170. The adjustments may be initiatedautomatically by controller 184 in response to one or more conditions.For example, controller 184 may instruct HVAC system 180 to providecooling to beam 174 when a temperature inside beam 174 exceeds apredetermined value. The adjustments may be initiated locally (e.g.,within vehicle 410) by a user of controller 184 (e.g., a technicianusing a computer that includes controller 184). The adjustments may beinitiated by controller 184 remotely (e.g., via a cellular or wirelesslink). The adjustments may include adjusting lighting system 178 (e.g.,a lighting orientation, a lighting intensity, etc.), adjusting HVACsystem 180 (e.g., temperature, humidity, etc.), adjusting a cameraorientation, and the like.

Controller 184 of image capturing system 170 may monitor a power statusof one or more components of image capturing system 170. Controller 184may provide power to one or more components (e.g., sensors of modules176) of image capturing system 170. Controller 184 may engage non-datafeatures such as remote reset. Controller 184 may receive a triggersignal from rotary position encoder 420 and distribute the signal to acontroller (e.g., a sensor controller) of module 176. The controller ofmodule 176 may then actuate one or more components of module 176 (e.g.,a sensor) in real-time with the control system signal. Image capturingsystem 170 may include one or more motion sensing sensors, such asgyroscopes and accelerometers, that provide a trigger signalcompensation factor to account for vehicle motions caused by extraneousfeatures (e.g., a rough track).

In certain embodiments, image capturing system 170 may receive a speedof vehicle 410. Controller 184 may receive the speed (e.g., a GPSvelocity signal) of from a GPS receiver, a radar speed measurementsystem, a laser speed measurement system, or any other suitablecomponent operable to measure speed of vehicle 410.

Although FIG. 4 illustrates a particular arrangement of vehicle 410,rotary position encoder 420, sub-frame 172, beam 174, modules 176,lighting system 178, HVAC system 180, data system 182, and controller184, this disclosure contemplates any suitable arrangement of vehicle410, rotary position encoder 420, sub-frame 172, beam 174, modules 176,lighting system 178, HVAC system 180, data system 182, and controller184. For example, lighting system 170 may be located externally to beam174. Vehicle 410, rotary position encoder 420, sub-frame 172, beam 174,modules 176, lighting system 178, HVAC system 180, data system 182, andcontroller 184 may be physically or logically co-located with each otherin whole or in part.

Although FIG. 4 illustrates a particular number of vehicles 410, rotaryposition encoders 420, sub-frames 172, beams 174, modules 176, lightingsystems 178, HVAC systems 180, data systems 182, and controllers 184,this disclosure contemplates any suitable number of vehicles 410, rotaryposition encoders 420, sub-frames 172, beams 174, modules 176, lightingsystems 178, HVAC systems 180, data systems 182, and controllers 180.For example, image capturing system 170 may include first beam 174 at afront end of vehicle 410 and second beam 174 at a rear end of vehicle410. As another example, image capturing system 178 may include multiplecontrollers 184 (e.g., a controller and a sub-controller). One or morecomponents of image capturing system 170 may be implemented using one ormore components of the computer system of FIG. 7 .

FIG. 5 illustrates an example module 176 that may be used by imagecapturing system 170 of FIG. 4 . Module 176 includes a camera 510, alens 520, a top plate 530, a base plate 540, a cover plate 550, acleaning device 560, bolts 570, an opening 580, and air compressor 585.Camera 510 is any device that captures images (e.g., images 152 of FIG.1 ) of physical objects. For example, camera 510 may capture images of arail component (e.g., a rail joint, a switch, a frog, a fastener,ballast, a rail head, and/or a rail tie). In certain embodiments, camera510 is a sensor.

One or more cameras 510 may capture images of one or more physicalobjects from different angles. For example, one or more cameras 510 maycapture images of both rails of a railway system at any given location.Each beam (e.g., beam 174 of FIG. 4 ) may include multiple cameras 510.The beam may include first camera 510 aimed straight down to capture anoverhead image of the physical object. The beam may include secondcamera 510 aimed downward and outward to capture an angled image of thephysical object.

Camera 510 may be a line scan camera. A line scan camera includes asingle row of pixels. Camera 510 may be a dual line scan camera. A dualline scan camera includes two rows of pixels that may be captured and/orprocessed simultaneously. As camera 510 moves over the physical object,camera 510 may capture images such that a complete image of the physicalobject can be reconstructed in software line by line. Camera 510 mayhave a capture rate up to 140 kilohertz. Camera 510 may have aresolution and optics to detect physical objects of at least 1/16 inchesin size. In alternative embodiments, camera 510 may be an area scancamera.

Camera 510 includes lens 520 that focuses and directs incident light toa sensor of camera 510. Lens 520 may be a piece of glass or othertransparent substance. Lens 520 may be made of any suitable material(e.g., steel, aluminum, glass, plastic, or a combination thereof.)

Top plate 530 and base plate 540 are structural elements used toposition, support, and/or stabilize one or more components of module 176(e.g., camera 510 or a sensor). Top plate 530 and bottom plate 540 maybe made of any suitable material (e.g., steel, aluminum, plastic, glass,and the like). Top plate 530 may be connected to base plate 540 with oneor more bolts 570. Bolts 570 (e.g., jack bolts) may be used to alter apitch and/or roll orientation of camera 510. For example, bolts 570 maybe used to change an effective height between top plate 530 and baseplate 540. Top plate 530 and/or base plate 540 may be adjusted to reducevibration and/or shock of module 176. Top plate 530 and/or base plate540 may include resistive heating elements to provide a warm environmentfor camera 510 and lens 520 to operate during cooler weather.

Cover plate 550 is a plate that covers base plate 540. Cover plate 550may be made of any suitable material (e.g., glass, steel, aluminum, andthe like). Cover plate 550 includes an opening 580. Opening 580 mayserve as an aperture through which a lens of camera 510 views thephysical object. Opening 580 allows for transmission of a sensed signalfrom the environment to reach a sensor of camera 510. Opening 580 may beany suitable size (e.g., oval, rectangular, and the like) to accommodateviews of camera 510. Lens 520 of camera 510 and/or air compressor 585may be positioned directly over opening 580. Air compressor 585 mayprovide cooling to beam 174 via an air-powered vortex generator asdescribed in FIG. 4 above.

Cleaning device 560 is any device that protects lens 520 and/or a sensorof camera 510 from the environment. Cleaning device 560 dislodges dust,small debris, water, and/or other items that may obstruct the viewthrough opening 580. Cleaning device 560 provides minimal or noobstruction of a signal transmitted by a component (e.g., a camera or asensor) of module 176. Cleaning device 560 may be located between coverplate 550 and based plate 540. In certain embodiments, cleaning device560 physically connects to cover plate 550 and/or base plate 540.Cleaning device 560 may be made of any suitable material (e.g., glass,steel, aluminum, and the like). Cleaning device 560 may be located on anexternal face of module 176 (e.g., an underside of module 176).

Cleaning device 560 may employ any suitable method to clean lens 520 ofcamera 510. For example, cleaning device 560 may include a cleaningagent that emits compressed air, compressed gas, or a cleaning fluid.Cleaning device 560 may include a wiper blade, a brush, or any othersuitable method to clean lens 520. In certain embodiments, the cleaningagent is a compressor (e.g., air compressor 585) that emits compressedair or compressed gas. The compressed air or compressed gas isdischarged through an orifice designed to utilize the Coanda effect,which entrains nearby air into the primary stream to amplify the amountof air (see notation 590 for air flow) displaced across lens 520. Incertain embodiments, cleaning device 560 may be part of an HVAC system(e.g., HVAC system 180 of FIG. 4 ).

Although FIG. 5 illustrates a particular arrangement of camera 510, lens520, top plate 530, base plate 540, cover plate 550, cleaning device560, bolts 570, opening 580, and air compressor 585, this disclosurecontemplates any suitable arrangement of camera 510, lens 520, top plate530, base plate 540, cover plate 550, cleaning device 560, bolts 570,opening 580, and air compressor 585. Although FIG. 5 illustrates aparticular number of cameras 510, lenses 520, top plates 530, baseplates 540, cover plates 550, cleaning devices 560, bolts 570, openings580, and air compressors 585, this disclosure contemplates any suitablenumber of cameras 510, lenses 520, top plates 530, base plates 540,cover plates 550, cleaning devices 560, bolts 570, openings 580, and aircompressors 585. For example, module 176 may include multiple cameras510. One or more components of module 176 may be implemented using oneor more components of the computer system of FIG. 7 .

FIG. 6 illustrates an example method 600 for tagging features 154 withlabels 162. FIG. 6 includes a diagram of a rail 630 and a rail joint bar632. Method 600 begins at step 610, where a classification engine (e.g.,classification engine 132 of FIG. 1 ) identifies features 154 of railjoint 132. Features 154 include a first square nut 634, a first bolthead 636, a first hole 638, a second hole 640, a second square nut 642,a second bolt head 636, a break 646, and a discontinuity 648. Break 646represents a defect (e.g., defect 156 of FIG. 1 ) of rail joint bar 632.Discontinuity 648 represents a separation between rails 630. Theclassification engine may identify (e.g., classify) features 154 of railjoint 132 using one or more CNNs.

At step 620 of method 600, a labeling engine (e.g., labeling engine 138of FIG. 1 ) tags features 154 of rail joint 132 with labels 162. Asshown, the labeling engine tags feature 634 as “Square nut (0.84),” tagsfeature 636 as “Bolt-head (0.77),” tags feature 638 as “Hole (0.89),”feature 640 as “Hole (0.89),” feature 642 as “Square nut (0.84),”feature 644 as “Bolt-head (0.77),” feature 646 as “Break (0.83),” andfeature 648 as “discontinuity (0.87).” Each label includes aclassification (e.g., classification 155 of FIG. 1 ) of each feature 154and a probability of each classification accurately identifying eachfeature. For example, feature 636, which is tagged as “Bolt-head(0.77),” represents a classification of “bolt-head” and a probability of0.77 (e.g., 77 percent) that “Bolt-head” accurately identifies feature636. As another example, feature 638, which is tagged as “Hole (0.89),”represents a classification of “hole” and a probability of 0.89 (e.g.,89 percent) that “hole” accurately identifies feature 638. As anotherexample, feature 646, which is tagged as “Break (0.83),” represents aclassification of “break” and a probability of 0.83 (e.g., 83 percent)that “break” accurately identifies feature 646. As still anotherexample, feature 648, which is tagged as “discontinuity (0.87),”represents a classification of “discontinuity” and a probability of 0.87(e.g., 87 percent) that “discontinuity” accurately identifies feature648.

The classifications and/or probabilities may be determined by theclassification engine using one or more machine learning networks (e.g.,CNNs) and/or algorithms. In the illustrated embodiment, the location ofeach bounding box represents a location of each feature 154 relative tothe other features 154 and relative to rail joint bar 632. Each boundingbox may be any suitable quadrilateral shape (i.e., a square, arectangle, etc.). An object detection model may be used to output thebounding boxes and labels 162 of the detected classifications. While theillustrated embodiment of FIG. 6 represents features 154 with boundingboxes, features 154 may be represented by any suitable shape and/orcharacter.

Method 600 may be used to further classify a defect. In FIG. 6 , thedefect is illustrated as a “Break.” One or more machine learningalgorithms (e.g., CNNs 166 of FIG. 1 ) may be trained to recognize theillustrated classifications of square nut, bolt-head, break, hole,discontinuity, and bar. While the one or more CNNs are trained to make avisual distinction between the square nuts and the bolt heads, thevisual representation of both classifications equates to the physicalrepresentation of the presence of a bolt. A defect detector engine(e.g., defect detector engine 134 of FIG. 1 ) may use one or morealgorithms to logically reduce the square nut and bolt-headclassifications to a single bolt classification. The defect detectorengine may use one or more algorithms to check the relative positions(e.g., locations 158 of FIG. 1 ) of each of the detected objects (i.e.,the bolts, the break, the holes, and the discontinuity).

The defect detector engine may use one or more algorithms to determinewhether the break detected in the joint bar is a center broken orquarter broken defect using the relative positions. In the illustratedembodiment, the detected bounding box for the break is positionedbetween the two middle bolts (i.e., “Bolt-head (0.77)” and “Square nut(0.84)”). The defect detector engine may determine that the break is acenter broken joint bar defect due to the position of the break betweenthe two middle bolts, which indicates that the break is in closeproximity to the discontinuity. In alternative embodiments, the breakmay be positioned outside of the two middle bolts, and the defectdetector engine may determine that the break is a quarter broken jointbar due to the position of the break outside the two middle bolts.

Modifications, additions, or omissions may be made to method 600depicted in FIG. 6 . Method 600 may include more, fewer, or other steps.For example, as discussed above, method 600 may include furtherclassifying the defect (e.g., a break) of the physical object (e.g., arail joint bar) into a more specific defect (e.g., a center broken jointbar or a quarter broken joint bar) based on a location of the defectrelative to other features (e.g., bolts, holes, and discontinuities) ofthe physical object. Steps may be performed in parallel or in anysuitable order. While discussed as specific components completing thesteps of method 600, any suitable component may perform any step ofmethod 600.

FIG. 7 shows an example computer system that may be used by the systemsand methods described herein. For example, network 110, defect detectormodule 120, and image capturing system 170 of FIG. 1 may include one ormore interface(s) 710, processing circuitry 720, memory(ies) 730, and/orother suitable element(s). Interface 710 (e.g., interface 122 of FIG. 1) receives input, sends output, processes the input and/or output,and/or performs other suitable operation. Interface 710 may comprisehardware and/or software.

Processing circuitry 720 (e.g., processor 126 of FIG. 1 ) performs ormanages the operations of the component. Processing circuitry 720 mayinclude hardware and/or software. Examples of a processing circuitryinclude one or more computers, one or more microprocessors, one or moreapplications, etc. In certain embodiments, processing circuitry 720executes logic (e.g., instructions) to perform actions (e.g.,operations), such as generating output from input. The logic executed byprocessing circuitry 720 may be encoded in one or more tangible,non-transitory computer readable media (such as memory 730). Forexample, the logic may comprise a computer program, software, computerexecutable instructions, and/or instructions capable of being executedby a computer. In particular embodiments, the operations of theembodiments may be performed by one or more computer readable mediastoring, embodied with, and/or encoded with a computer program and/orhaving a stored and/or an encoded computer program.

Memory 730 (or memory unit) stores information. Memory 730 (e.g., memory124 of FIG. 1 ) may comprise one or more non-transitory, tangible,computer-readable, and/or computer-executable storage media. Examples ofmemory 730 include computer memory (for example, RAM or ROM), massstorage media (for example, a hard disk), removable storage media (forexample, a Compact Disk (CD) or a Digital Video Disk (DVD)), databaseand/or network storage (for example, a server), and/or othercomputer-readable medium.

Herein, a computer-readable non-transitory storage medium or media mayinclude one or more semiconductor-based or other integrated circuits(ICs) (such as field-programmable gate arrays (FPGAs) orapplication-specific ICs (ASICs)), hard disk drives (HDDs), hybrid harddrives (HHDs), optical discs, optical disc drives (ODDs),magneto-optical discs, magneto-optical drives, floppy diskettes, floppydisk drives (FDDs), magnetic tapes, solid-state drives (SSDs),RAM-drives, SECURE DIGITAL cards or drives, any other suitablecomputer-readable non-transitory storage media, or any suitablecombination of two or more of these, where appropriate. Acomputer-readable non-transitory storage medium may be volatile,non-volatile, or a combination of volatile and non-volatile, whereappropriate.

Herein, “or” is inclusive and not exclusive, unless expressly indicatedotherwise or indicated otherwise by context. Therefore, herein, “A or B”means “A, B, or both,” unless expressly indicated otherwise or indicatedotherwise by context. Moreover, “and” is both joint and several, unlessexpressly indicated otherwise or indicated otherwise by context.Therefore, herein, “A and B” means “A and B, jointly or severally,”unless expressly indicated otherwise or indicated otherwise by context.

The scope of this disclosure encompasses all changes, substitutions,variations, alterations, and modifications to the example embodimentsdescribed or illustrated herein that a person having ordinary skill inthe art would comprehend. The scope of this disclosure is not limited tothe example embodiments described or illustrated herein. Moreover,although this disclosure describes and illustrates respectiveembodiments herein as including particular components, elements,feature, functions, operations, or steps, any of these embodiments mayinclude any combination or permutation of any of the components,elements, features, functions, operations, or steps described orillustrated anywhere herein that a person having ordinary skill in theart would comprehend. Furthermore, reference in the appended claims toan apparatus or system or a component of an apparatus or system beingadapted to, arranged to, capable of, configured to, enabled to, operableto, or operative to perform a particular function encompasses thatapparatus, system, component, whether or not it or that particularfunction is activated, turned on, or unlocked, as long as thatapparatus, system, or component is so adapted, arranged, capable,configured, enabled, operable, or operative. Additionally, although thisdisclosure describes or illustrates particular embodiments as providingparticular advantages, particular embodiments may provide none, some, orall of these advantages.

What is claimed is:
 1. A method, comprising: receiving, by a defectdetector module, an image of a physical object; classifying, by thedefect detector module, one or more first features from the image of thephysical object into one or more first classifications using one or moremachine learning algorithms; analyzing, by the defect detector module,the one or more first classifications; determining, by the defectdetector module, that the physical object comprises a defect based onanalyzing the one or more first classifications; determining, by thedefect detector module, a location of the defect within the image;determining, by the defect detector module, that the location of thedefect within the image is part of the physical object; and determining,by the defect detector module, a geographic location of the defect ofthe physical object based at least in part on the location of the defectwithin the image, wherein determining the geographic location of thedefect of the physical object comprises determining a latitude and alongitude associated with the image.
 2. The method of claim 1, whereinthe defect detector module translates the latitude and longitude into atleast one of a track type, a track number, a line segment, or amilepost.
 3. The method of claim 1, further comprising labeling, by thedefect detector module, the one or more first features of the image withone or more labels.
 4. The method of claim 1, further comprisingtraining, by the defect detector module, the one or more machinelearning algorithms to classify the one or more first features from theimage by collecting sample data representative of the one or more firstfeatures.
 5. The method of claim 1, wherein: the physical object is arail joint; the defect is a broken rail joint; and the one or more firstclassifications comprise one or more of the following: a bolt; a break;a hole; and a discontinuity.
 6. The method of claim 1, furthercomprising: classifying, by the defect detector module, one or moresecond features from the image of the physical object into one or moresecond classifications using the one or more machine learningalgorithms, wherein the one or more second classifications comprise atleast one of the following: a bar, a discontinuity, and an end post; andcropping, by the defect detector module, the image to an areasurrounding the one or more second features to create a cropped image;wherein classifying the one or more first features from the image of thephysical object into one or more first classifications comprisesclassifying the one or more first features from the cropped image of thephysical object into one or more first classifications.
 7. The method ofclaim 1, wherein the image of the physical object is captured by acomponent in motion relative to the physical object.
 8. A systemcomprising one or more processors and a memory storing instructionsthat, when executed by the one or more processors, cause the one or moreprocessors to perform operations comprising: receiving, by a defectdetector module, an image of a physical object; classifying, by thedefect detector module, one or more first features from the image of thephysical object into one or more first classifications using one or moremachine learning algorithms; analyzing, by the defect detector module,the one or more first classifications; determining, by the defectdetector module, that the physical object comprises a defect based onanalyzing the one or more first classifications; determining, by thedefect detector module, a location of the defect within the image;determining, by the defect detector module, that the location of thedefect within the image is part of the physical object; and determining,by the defect detector module, a geographic location of the defect ofthe physical object based at least in part on the location of the defectwithin the image, wherein determining the geographic location of thedefect of the physical object comprises determining a latitude and alongitude associated with the image.
 9. The system of claim 8, whereinthe defect detector module translates the latitude and longitude into atleast one of a track type, a track number, a line segment, or amilepost.
 10. The system of claim 8, the operations further comprisinglabeling, by the defect detector module, the one or more first featuresof the image with one or more labels.
 11. The system of claim 8, theoperations further comprising training, by the defect detector module,the one or more machine learning algorithms to classify the one or morefirst features from the image by collecting sample data representativeof the one or more first features.
 12. The system of claim 8, wherein:the physical object is a rail joint; the defect is a broken rail joint;and the one or more first classifications comprise one or more of thefollowing: a bolt; a break; a hole; and a discontinuity.
 13. The systemof claim 8, the operations further comprising: classifying, by thedefect detector module, one or more second features from the image ofthe physical object into one or more second classifications using theone or more machine learning algorithms, wherein the one or more secondclassifications comprise at least one of the following: a bar, adiscontinuity, and an end post; and cropping, by the defect detectormodule, the image to an area surrounding the one or more second featuresto create a cropped image; wherein classifying the one or more firstfeatures from the image of the physical object into one or more firstclassifications comprises classifying the one or more first featuresfrom the cropped image of the physical object into one or more firstclassifications.
 14. The system of claim 8, wherein the image of thephysical object is captured by a component in motion relative to thephysical object.
 15. One or more non-transitory storage media embodyinginstructions that, when executed by a processor, cause the processor toperform operations comprising: receiving, by a defect detector module,an image of a physical object; classifying, by the defect detectormodule, one or more first features from the image of the physical objectinto one or more first classifications using one or more machinelearning algorithms; analyzing, by the defect detector module, the oneor more first classifications; determining, by the defect detectormodule, that the physical object comprises a defect based on analyzingthe one or more first classifications; determining, by the defectdetector module, a location of the defect within the image; determining,by the defect detector module, that the location of the defect withinthe image is part of the physical object; and determining, by the defectdetector module, a geographic location of the defect of the physicalobject based at least in part on the location of the defect within theimage, wherein determining the geographic location of the defect of thephysical object comprises determining a latitude and a longitudeassociated with the image.
 16. The one or more non-transitory storagemedia of claim 15, wherein the defect detector module translates thelatitude and longitude into at least one of a track type, a tracknumber, a line segment, or a milepost.
 17. The one or morenon-transitory storage media of claim 15, the operations furthercomprising labeling, by the defect detector module, the one or morefirst features of the image with one or more labels.
 18. The one or morenon-transitory storage media of claim 15, the operations furthercomprising training the one or more machine learning algorithms toclassify the one or more first features from the image by collectingsample data representative of the one or more first features.
 19. Theone or more non-transitory storage media of claim 15, wherein: thephysical object is a rail joint; the defect is a broken rail joint; andthe one or more first classifications comprise one or more of thefollowing: a bolt; a break; a hole; and a discontinuity.
 20. The one ormore non-transitory storage media of claim 15, the operations furthercomprising: classifying, by the defect detector module, one or moresecond features from the image of the physical object into one or moresecond classifications using the one or more machine learningalgorithms, wherein the one or more second classifications comprise atleast one of the following: a bar, a discontinuity, and an end post; andcropping, by the defect detector module, the image to an areasurrounding the one or more second features to create a cropped image;wherein classifying the one or more first features from the image of thephysical object into one or more first classifications comprisesclassifying the one or more first features from the cropped image of thephysical object into one or more first classifications.